Searching for images in a large database has in the past been very cumbersome. Content-based image retrieval, also known as query by image content and content-based visual information retrieval, is the application of computer vision to the image retrieval problem, that is, the problem of searching for digital images in large databases. “Content-based” means that the search makes use of the contents of the images themselves, rather than relying on metadata such as captions or keywords, which is typically input by humans.
There is growing interest in content-based image retrieval because of the limitations inherent in meta data-based systems. Textual information about images can be easily searched using existing technologies, but this type of a search requires humans to personally describe every image in the database. This is impractical for very large databases, or for images that are generated automatically, such as, for example, from surveillance cameras. It is also possible to miss images that use different synonyms in their descriptions.
The ideal content-based image retrieval system from a user's perspective involves semantic retrieval, where the user makes a request like “find pictures of a flower”. This type of open-ended task is very difficult for computers to perform—pictures of roses and lilies look very different. Current content-based information systems therefore generally make use of lower level features like texture, color, and shape in searching for images, although some systems take advantage of higher-level features.
Object instance (or known object) recognition is one type of content-based image retrieval. Object instance recognition is the task of recognizing a specific object in an image. Object instance recognition does not recognize categories of objects, but instead a particular object from a category. By way of example, these specific objects may include specific artwork (such as the Mona Lisa), a specific photograph, the front of a specific restaurant, or an object on a supermarket shelf.
Major search engines have yet to implement content based image retrieval and object instance recognition image retrieval to browse or search through their indexes of images, the largest of which contain links to billions of photographs and graphics. Still, research by both industry and academia has achieved some intriguing advances of late that sidestep the need for keywords—and address the challenge of analyzing the content of images in large databases.